The invention relates to image processing systems, and more particularly, to apparatus and a concomitant method of detecting motion within an image sequence.
Motion detection is used in various forms of surveillance and monitoring systems. These systems may detect motion using rudimentary sensors such as ultra-sound and infrared sensors, where any change in the background audio or infrared signal is interpreted as motion. Signal processing in these systems can be as simple as a threshold level that, when exceeded, indicates motion within the sensor""s range.
On the other hand, surveillance and monitoring systems may rely upon complex video image processing to analyze a series of images captured by a video camera. Processing of the images requires frame-to-frame comparison of the image content, usually accomplished by subtracting a given frame from a previous frame, resulting in a difference image representing changes between the two images. The absolute value or the square of the difference image results in an image that represents the magnitude of change at each position in the image. This magnitude information alone is useful in determining if motion has occurred within a scene, but is not indicative of the direction of the motion or computing motion xe2x80x9cflowxe2x80x9d between successive images.
Determination of the direction of the motion is generally accomplished by generating multiple images of magnitude information, e.g., a magnitude information image sequence, and determining the movement of the magnitude information within the image sequence. This can be accomplished by tracking areas in the image sequence having significant change or change that conforms to a specific criteria such a having a size, shape and speed of a person walking or running through the scene. To determine the specific motion direction, the system must independently track the physical position if any significant change that occurs from one image to the next. Independently tracking this movements can be a daunting process for any reasonably priced computer system. Consequently, these systems are prone to false alarms when faced with complex motion such a walking person or multiple moving objects within the scene.
To limit the potential number of areas that are to be independently tracked, and thus reduce the computational complexity required to track movement in the scene, the typical surveillance and monitoring system images a restricted scene. In the restricted scene, the position, size and direction of motion of most of the xe2x80x9cnormalxe2x80x9d motion in the scene is proscribed by physical barriers. For example, in an airport surveillance system, physical barriers are used to channel all persons through a single area in the scene. As such, all xe2x80x9cnormalxe2x80x9d motion is through the channel and in a single direction. Abnormal motion is defined as motion within the scene, that is outside the channel. Such motion would occur if a person were to xe2x80x9cjumpxe2x80x9d the barrier. The use of image optical flow-based methods for determining motion in an unrestricted scene is not viable using traditional techniques because such methods are prone to error, especially in areas of complex motion. This problem keeps flow-based methods from being a viable alternative for motion discrimination without significant restriction of image motion. The use of change-based methods without flow are also noise-prone, especially when changes in illumination of the scene are present.
Such restrictive systems are useless in many security applications and can be overly restrictive to customers and passengers. Consequently, use of such systems has not been widespread.
Therefore, a need exists in the art for a method and apparatus that accurately detects motion within an image sequence and, more particularly, a method and apparatus that detects both the magnitude and the direction of the motion without the need for restricting the motion in the scene.
The disadvantages heretofore associated with prior art motion direction detection are overcome by the present invention. The invention is apparatus and a concomitant method of identifying the direction of object motion within a scene that is represented by a sequence of images, e.g., a series of frames in a video sequence. The invention processes the sequence of images and generates a substantially distortionless flow field representing the motion within a scene. The flow field is a vector representation of the motion of the scene that represents both the magnitude and the direction of the motion. The flow field is generated by sequentially correlating two or more frames in the sequence of images, using shaping pre-filters and specialized correlation analysis techniques to create a flow field with high fidelity and noise suppression.
The flow field of the present scene is analyzed by a flow field segmentor. The segmentor determines the magnitude and direction of motion with the scene and segments the motion information from the static portion of the image sequence. An alarm detector then processes the motion information to determine if an alarm should be generated based upon the motion information. Specifically, the alarm detector compares the motion information to an alarm criteria. If the motion information fits the criteria, an alarm flag is sent to the alarm processor. The alarm processor activates an appropriate alarm response, e.g., illuminate warning lamps, sound a warning alarm, lock doors, alert security personnel, and the like.